# Machine Perception

Project Aria is a Meta project focused on first-person perspective research, aiming to advance augmented reality (AR) and artificial intelligence (AI) through innovative technologies. The project collects information from a user's perspective using devices such as the Aria Gen 2 glasses, providing support for machine perception and AR research. Its key advantages include innovative hardware design, rich open-source datasets and challenges, and close collaboration with global research partners. The project's background stems from Meta's long-term investment in future AR technology, aiming to drive industry progress through open research.

Aria Gen 2 is Meta's second-generation research-grade smart glasses, designed specifically for machine perception, contextual AI, and robotics research. It integrates advanced sensors and low-power machine perception technology, capable of real-time processing of SLAM, eye tracking, and gesture recognition. This product aims to advance artificial intelligence and machine perception technologies, providing researchers with powerful tools to explore how AI can better understand the world from a human perspective. Aria Gen 2 not only achieves technological breakthroughs but also promotes open research and public understanding of these crucial technologies through collaborations with academia and commercial research labs.

Alchemist is a technology that utilizes a pre-trained text-to-image model and synthesized data to allow users to edit the material properties of objects in images. It achieves parameterized control over specific material properties, such as roughness, metallicity, base color saturation, and transparency, through fine-tuning on synthesized datasets. The main advantages of this technology include the ability to alter object properties while maintaining the geometric shape and lighting of the image, and even realistically filling in backgrounds, hidden internal structures, and refraction effects when objects become transparent.

Ego-Exo4D is a multimodal, multi-view video dataset and benchmark challenge focused on capturing first-person and external perspectives of skill-based human activities. It supports multi-modal machine perception research for daily activities. The dataset was collected by 839 volunteers wearing cameras in 13 cities worldwide, capturing 1422 hours of skill-based human activity videos. The dataset provides three types of paired video-aligned natural language datasets: expert annotations, participant-provided tutorial-style narratives, and one-sentence atomic action descriptions. Ego-Exo4D also captures multi-view and multi-sensory modalities, including multiple cameras, seven microphone arrays, two IMUs, a barometer, and a magnetometer. The dataset was recorded strictly adhering to privacy and ethical policies with informed consent from participants. For more information, please visit the official website.